Papers with NLP pipelines
Latent Structure Models for Natural Language Processing (P19-4)
Copied to clipboard
| Challenge: | Latent structure models are a powerful tool for compositional data modeling and pipelines. |
| Approach: | This tutorial will cover recent advances in discrete latent structure models . it will discuss their motivation, potential, and limitations . |
| Outcome: | This tutorial will cover recent advances in discrete latent structure models . it will discuss their motivation, potential, and limitations . |
MaintNet: A Collaborative Open-Source Library for Predictive Maintenance Language Resources (2020.coling-demos)
Copied to clipboard
| Challenge: | Maintenance record logbooks are an emerging text type in NLP. maintenance record logbook data is often written in non-standard language with many domain specific technical terms, abbreviations, and non-standardized spelling and grammar. |
| Approach: | They propose to create a collaborative open-source library of technical and domain-specific language resources for maintenance record logbooks. |
| Outcome: | The proposed library provides tools to aid in their (pre-)processing and clustering. |
NLP Tools for Predictive Maintenance Records in MaintNet (2020.aacl-demo)
Copied to clipboard
| Challenge: | Maintenance logbooks often contain free text fields with domain specific terms, abbreviations, and non-standard spelling . most standard NLP pipelines for pre-processing and annotation are trained on standard contemporary corpora. |
| Approach: | They propose to create an open-source library and data repository for predictive maintenance language datasets and to evaluate the tools available at MaintNet. |
| Outcome: | The proposed tools improve the performance of existing pipelines and improve the quality of the existing ones. |
Erase and Rewind: Manual Correction of NLP Output through a Web Interface (2021.acl-demo)
Copied to clipboard
| Challenge: | In the last years, NLP tools are being used in tasks such as textual inference, machine translation, hate speech detection. |
| Approach: | They propose an NLP annotation software that can be used to manually annotate texts and to fix mistakes in NLP pipelines. |
| Outcome: | The proposed tool can be used to manually annotate texts and fix errors in NLP pipelines, such as Stanford CoreNLP. |
BENTO: A Visual Platform for Building Clinical NLP Pipelines Based on CodaLab (2020.acl-demos)
Copied to clipboard
| Challenge: | CodaLab has limited support for creating reusable tools that can be easily applied to different datasets and composed into pipelines. |
| Approach: | They propose a workflow management platform with a graphic user interface built on top of CodaLab to facilitate the process of building clinical NLP pipelines. |
| Outcome: | The proposed workflow management platform, BENTO, is designed for clinical NLP tasks and can be easily used by researchers and developers. |
Vocabulary Matters: A Simple yet Effective Approach to Paragraph-level Question Generation (2020.aacl-main)
Copied to clipboard
| Challenge: | Current neural network-based questions generation techniques take only one or two sentences as input. |
| Approach: | They propose a simple yet effective technique for question generation from paragraphs . they augment a sequence-to-sequence QG model with dynamic, paragraph-specific dictionary . |
| Outcome: | The proposed model outperforms state-of-the-art systems in question generation from paragraphs in automatic and human evaluation. |
FastSense: An Efficient Word Sense Disambiguation Classifier (L18-1)
Copied to clipboard
| Challenge: | Word Sense Disambiguation (WSD) is a task that is often overlooked by NLP pipelines because of its complexity and complexity. |
| Approach: | They propose a neural network-based tool for word sense disambiguation called fastSense. |
| Outcome: | The proposed tool can process huge amounts of data quickly and surpasses state-of-the-art tools in terms of F-measure. |
Laying Anchors: Semantically Priming Numerals in Language Modeling (2024.findings-naacl)
Copied to clipboard
| Challenge: | Numeracy is the comprehension of numbers, and numerals are important for comprehension. |
| Approach: | They propose methods to semantically prime numerals by generating anchors governed by the distribution of numeral in any corpus. |
| Outcome: | The proposed methods perform better on numeracy tasks for both in-domain and out-domain numerals. |
Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little (2021.emnlp-main)
Copied to clipboard
| Challenge: | masked language models (MLMs) pre-train to model higher-order word co-occurrence statistics . authors suggest that such models have learned to represent syntactic structures prevalent in classical NLP pipelines . purely distributional information largely explains the success of pre-training, authors say . |
| Approach: | They propose to pre-train masked language models on sentences with random shuffled word order and show they still achieve high accuracy after fine-tuning on many downstream tasks. |
| Outcome: | The proposed model performs well according to parametric syntactic probes . the authors argue that the model is not all that different from earlier distributional models . |
Tokenization and the Noiseless Channel (2023.acl-long)
Copied to clipboard
| Challenge: | Subword tokenization is a key part of most NLP pipelines, but little is known about why some combinations lead to improved downstream model performance. |
| Approach: | They propose that good tokenizers lead to efficient channel usage . they propose that an optimal encoding assigns extremely long codes to low-frequency subwords . |
| Outcome: | The proposed tokenizers have a very strong correlation with BLEU in machine translation . the proposed function can be used to improve model performance in the downstream task . |
GenderQuant: Quantifying Mention-Level Genderedness (N19-1)
Copied to clipboard
| Challenge: | Existing approaches to detect gendered language require considerable annotation efforts for each language, domain, and author, and often require handcrafted lexicons and features. |
| Approach: | They use existing NLP pipelines to automatically annotate gender of mentions in the text and train a supervised classifier to predict the gender of any mention from its context and evaluate it on unseen text. |
| Outcome: | The proposed method can detect gendered language on movie summaries, movie reviews, news articles, and fiction novels. |
A unified approach to sentence segmentation of punctuated text in many languages (2021.acl-long)
Copied to clipboard
| Challenge: | Existing tools for segmenting punctuated text in many languages are limited in their language coverage and evaluation is ad hoc. |
| Approach: | They propose a new context-based modeling approach that can be trained on noisily-annotated data. |
| Outcome: | The proposed model exceeds baselines set by existing methods on English corpora and performs well on average on new multilingual evaluation set. |
Handling Extreme Class Imbalance in Technical Logbook Datasets (2021.acl-long)
Copied to clipboard
| Challenge: | Technical logbooks are a challenging and under-explored text type in automated event identification. |
| Approach: | They propose a feedback strategy that resamples the training data based on its error in the prediction process. |
| Outcome: | The proposed approach provides the best results for four different neural network models trained across a suite of technical logbook datasets from distinct technical domains. |
Parity-Aware Byte-Pair Encoding: Improving Cross-lingual Fairness in Tokenization (2026.acl-long)
Copied to clipboard
Negar Foroutan, Clara Meister, Debjit Paul, Joel Niklaus, Sina Ahmadi, Antoine Bosselut, Rico Sennrich
| Challenge: | Tokenization is the first step of most NLP pipelines. |
| Approach: | They propose a parity-aware byte pair encoder that maximizes the compression gain of the currently worst-compressed language for cross-lingual parity. |
| Outcome: | a new algorithm reduces tokenization inequality by 89% compared to classical BPE . the proposed algorithm is based on a fair-max rule that maximizes the compression gain of the currently worst-compressed language . |
Invariant Language Modeling (2022.emnlp-main)
Copied to clipboard
Maxime Peyrard, Sarvjeet Ghotra, Martin Josifoski, Vidhan Agarwal, Barun Patra, Dean Carignan, Emre Kiciman, Saurabh Tiwary, Robert West
| Challenge: | Existing methods to remove spurious correlations and biases involve expensive domain alignment. |
| Approach: | They propose a framework for learning invariant representations that generalize better across environments . they adapt a game-theoretic implementation of IRM to language models . |
| Outcome: | The proposed framework can remove structured noise, ignore correlations and achieve better generalization across environments. |
Social Biases in NLP Models as Barriers for Persons with Disabilities (2020.acl-main)
Copied to clipboard
| Challenge: | toxicity prediction and sentiment analysis models perpetuate undesirable social biases from the data on which they are trained. |
| Approach: | They propose to use toxicity prediction and sentiment analysis to examine whether NLP models perpetuate undesirable biases towards mentions of disability. |
| Outcome: | The proposed models contain undesirable biases towards mentions of disability in two English language models. |
The Role of Natural Language Processing Tasks in Automatic Literary Character Network Construction (2025.coling-main)
Copied to clipboard
| Challenge: | low-level tasks are used to extract character networks from literary texts, but no study has been conducted on their impact on performance. |
| Approach: | They focus on the role of named entity recognition (NER) and coreference resolution when extracting co-occurrence networks. |
| Outcome: | The proposed methods outperform traditional pipelines in terms of recall and recall. |
Debiasing knowledge graph embeddings (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to train knowledge graph embeddings to be neutral to sensitive attributes such as gender have been shown to increase training time by a factor of eight or more. |
| Approach: | They propose a method where all embeddings are trained to be neutral to sensitive attributes such as gender by default using an adversarial loss. |
| Outcome: | The proposed method reduces training time by eightfold and improves accuracy. |
Statistical Depth for Ranking and Characterizing Transformer-Based Text Embeddings (2023.emnlp-main)
Copied to clipboard
| Challenge: | Generalized transformer-based text embedding models have produced state of the art performance results on a variety of tasks such as natural language inference (NLI) |
| Approach: | They propose a statistical depth to measure distributions of transformer-based text embeddings and an associated rank sum test to characterize distributions in synthetic and human-generated corpora. |
| Outcome: | The proposed method improves performance over baseline methods on six text classification tasks. |
One Model is All You Need: ByT5-Sanskrit, a Unified Model for Sanskrit NLP Tasks (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Morphologically rich languages are notoriously challenging to process for downstream NLP applications. |
| Approach: | They propose a pretrained model for NLP applications involving the morphologically rich language Sanskrit that outperforms previous models by a considerable margin. |
| Outcome: | The proposed model outperforms tokenized models on established Sanskrit word segmentation tasks and matches the current best lexicon-based model. |
The xtsv Framework and the Twelve Virtues of Pipelines (2020.lrec-1)
Copied to clipboard
| Challenge: | Numerous small independent tools do not fit in a larger framework, and they may not be acknowledged properly. |
| Approach: | They propose an abstract framework for building NLP pipelines which is based on xtsv and a tsvo format with header plus some additional features. |
| Outcome: | The proposed framework is based on a concrete application for a Hungarian NLP pipeline. |
CASE: Efficient Curricular Data Pre-training for Building Assistive Psychology Expert Models (2024.findings-emnlp)
Copied to clipboard
Sarthak Harne, Monjoy Choudhury, Madhav Rao, T Srikanth, Seema Mehrotra, Apoorva Vashisht, Aarushi Basu, Manjit Sodhi
| Challenge: | Existing methods to identify mental health disorders rely on limited availability of psychologists. |
| Approach: | They propose to use forum posts to analyze text data to identify mental health issues . they propose to utilize readily available curricular texts for pre-training pipelines . |
| Outcome: | The proposed pipelines achieve an f1 score of 0.91 for Depression and 0.88 for Anxiety compared to existing pipelines. |
D3CODE: Disentangling Disagreements in Data across Cultures on Offensiveness Detection and Evaluation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies on annotator subjectivity focus on Western contexts and only document differences across age, gender, or racial groups. |
| Approach: | They propose a large-scale cross-cultural dataset of parallel annotations for offensive language in over 4.5K English sentences annotated by a pool of more than 4k annotators from 21 countries. |
| Outcome: | The proposed dataset captures annotators’ moral values along six moral foundations: care, equality, proportionality, authority, loyalty, and purity. |
Rethinking NLP for Chemistry: A Critical Look at the USPTO Benchmark (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Natural Language Processing (NLP) has revolutionized computer-aided synthesis planning by reframing chemical synthesis prediction as a sequence-to-sequence modeling problem over molecular string representations like SMILES. |
| Approach: | They propose to reframe chemical synthesis prediction as a sequence-to-sequence modeling problem over molecular string representations like SMILES. |
| Outcome: | The proposed framework yields impressive benchmark scores on the USPTO dataset, a large corpus of reactions extracted from US patents. |
Creating Grammar Teaching Material for Endangered Languages with Hybrid Grammar Induction (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for creating grammar lessons are labor-intensive and often fall to teachers who lack formal training in grammar. |
| Approach: | They propose a hybrid grammar-induction method that uses typological priors, Bayesian inference, constrained LLM reasoning and retrieval from sparse corpora to generate topic-specific grammar lessons. |
| Outcome: | The proposed method can produce coherent and useful lessons with better quality when modest explanatory evidence is available. |
ImpliRet: Benchmarking the Implicit Fact Retrieval Challenge (2025.emnlp-main)
Copied to clipboard
| Challenge: | Retrieval systems rely on surface-level cues such as keyword overlap and semantic similarity to evaluate retrieval beyond these shallow signals. |
| Approach: | They propose a benchmark that shifts the reasoning challenge to query-side processing techniques that can help resolve complexity. |
| Outcome: | The proposed benchmarks show that document-side reasoning remains a challenge. |